Systems and methods for providing contextually relevant information

ABSTRACT

Provided herein are methods and systems for enabling users to provide contextually relevant information, such as feedback, relating to a content item. A computing device may receive a request for a content item. The computing device may receive the request and analyze a first portion of the content item to determine contextual information associated with the first portion. The computing device may determine one or more suggestions for contextually relevant items, such as symbols, to enable use of the symbols to provide information. The computing device may send the one or more suggestions with the first portion of the content item.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.17/188,674 filed on Mar. 1, 2021, which is incorporated herein in itsentirety.

BACKGROUND

Viewers of content items may desire to share feedback while viewing acontent item. For example, a viewer may wish to send a message whileviewing a content item. Existing solutions do not provide contextuallyrelevant information options that are contextually relevant. These andother considerations are described herein.

SUMMARY

It is to be understood that both the following general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive. Provided herein are methods and systems forproviding contextually relevant items that enable feedback related to acontent item being viewed at a user device (e.g., a media player,smartphone, television, etc.). One or more contextual modules, such asan audio analysis module, a visual analysis module, a closed captionsmodule, or a commentary module may analyze a first portion of thecontent item and determine contextual information and/or aclassification associated with the first portion (e.g., a sceneclassification). One or more suggestions for contextually relevantinformation (e.g., emojis, messages, etc.) may be determined based onthe contextual information and/or the classification associated with thefirst portion.

The one or more suggestions may be provided to the user device alongwith the first portion of the content item. The user device may output(e.g., display/show/present) the one or more suggestions and the firstportion of the content item. For example, the one or more suggestionsmay be output at the user device as an overlay. A user of the userdevice may select at least one of the one or more suggestions duringoutput of the first portion (e.g., while the first portion is beingdisplayed). The user device may send an indication of the selection ofthe at least one suggestion to a computing device, such as a server. Thecomputing device may store the indication of the user's selection asmetadata. The computing device may use the metadata when determining(e.g., selecting) one or more suggestions for contextually relevantinformation for the user, another user, etc. Other examples andconfigurations are possible. Additional advantages will be set forth inpart in the description which follows or may be learned by practice. Theadvantages will be realized and attained by means of the elements andcombinations particularly pointed out in the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the present description serve to explain the principles of themethods and systems described herein:

FIG. 1 shows an example system;

FIG. 2 shows an example module;

FIGS. 3A and 3B show example graphs;

FIGS. 4A and 4B show example computing devices;

FIGS. 5A and 5B show example computing devices;

FIG. 6 shows an example system;

FIG. 7 shows a flowchart for an example method;

FIG. 8 shows a flowchart for an example method;

FIG. 9 shows a flowchart for an example method; and

FIG. 10 shows a flowchart of an example method.

DETAILED DESCRIPTION

As used in the specification and the appended claims, the singular forms“a,” “an,” and “the” include plural referents unless the context clearlydictates otherwise. Ranges may be expressed herein as from “about” oneparticular value, and/or to “about” another particular value. When sucha range is expressed, another configuration includes from the oneparticular value and/or to the other particular value. Similarly, whenvalues are expressed as approximations, by use of the antecedent“about,” it will be understood that the particular value forms anotherconfiguration. It will be further understood that the endpoints of eachof the ranges are significant both in relation to the other endpoint,and independently of the other endpoint.

“Optional” or “optionally” means that the subsequently described eventor circumstance may or may not occur, and that the description includescases where said event or circumstance occurs and cases where it doesnot.

Throughout the description and claims of this specification, the word“comprise” and variations of the word, such as “comprising” and“comprises,” means “including but not limited to,” and is not intendedto exclude, for example, other components, integers or steps.“Exemplary” means “an example of” and is not intended to convey anindication of a preferred or ideal configuration. “Such as” is not usedin a restrictive sense, but for explanatory purposes.

It is understood that when combinations, subsets, interactions, groups,etc. of components are described that, while specific reference of eachvarious individual and collective combinations and permutations of thesemay not be explicitly described, each is specifically contemplated anddescribed herein. This applies to all parts of this applicationincluding, but not limited to, steps in described methods. Thus, ifthere are a variety of additional steps that may be performed it isunderstood that each of these additional steps may be performed with anyspecific configuration or combination of configurations of the describedmethods.

As will be appreciated by one skilled in the art, hardware, software, ora combination of software and hardware may be implemented. Furthermore,a computer program product on a computer-readable storage medium (e.g.,non-transitory) having processor-executable instructions (e.g., computersoftware) embodied in the storage medium may be implemented. Anysuitable computer-readable storage medium may be utilized including harddisks, CD-ROMs, optical storage devices, magnetic storage devices,memresistors, Non-Volatile Random Access Memory (NVRAM), flash memory,or a combination thereof.

Throughout this application reference is made to block diagrams andflowcharts. It will be understood that each block of the block diagramsand flowcharts, and combinations of blocks in the block diagrams andflowcharts, respectively, may be implemented by processor-executableinstructions. These processor-executable instructions may be loaded ontoa general purpose computer, special purpose computer, or otherprogrammable data processing apparatus to produce a machine, such thatthe processor-executable instructions which execute on the computer orother programmable data processing apparatus create a device forimplementing the functions specified in the flowchart block or blocks.

These processor-executable instructions may also be stored in acomputer-readable memory that may direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the processor-executable instructions stored in thecomputer-readable memory produce an article of manufacture includingprocessor-executable instructions for implementing the functionspecified in the flowchart block or blocks. The processor-executableinstructions may also be loaded onto a computer or other programmabledata processing apparatus to cause a series of operational steps to beperformed on the computer or other programmable apparatus to produce acomputer-implemented process such that the processor-executableinstructions that execute on the computer or other programmableapparatus provide steps for implementing the functions specified in theflowchart block or blocks.

Blocks of the block diagrams and flowcharts support combinations ofdevices for performing the specified functions, combinations of stepsfor performing the specified functions and program instruction means forperforming the specified functions. It will also be understood that eachblock of the block diagrams and flowcharts, and combinations of blocksin the block diagrams and flowcharts, may be implemented by specialpurpose hardware-based computer systems that perform the specifiedfunctions or steps, or combinations of special purpose hardware andcomputer instructions.

Provided herein are methods and systems to enable users to providecontextually relevant information, such as feedback, comments, thoughts,emotions, etc. In one implementation, users of computing devices (e.g.,user devices) may desire to provide feedback relating to a content itemthey are presently viewing. Feedback may include a message, a picture,an emoji, comments, and/or the like, and may be provided using symbols,text, audio and/or video. In order to provide contextually relevantinformation related to the content item, a computing device may analyzethe content item. As an example, a first user of a first user device maywish to view a content item. The user device may send a request for thecontent item to a computing device, such as a server or cache of acontent distribution network. The computing device may receive therequest and analyze a first portion of the content item. The firstportion may be a fragment, a segment, a scene, etc. The computing devicemay analyze the first portion using one or more contextual modules, suchas an audio analysis module, a visual analysis module, a closed captionsmodule, or a commentary module.

The computing device may analyze the first portion to determinecontextual information associated with the first portion. For example,the computing device may use the one or more contextual modules todetermine the contextual information based on at least one of: an audiocomponent of the first portion, a visual component of the first portion,a textual component of the first portion, or commentary data associatedwith the first portion. An audio component of the first portion mayinclude one or more words spoken, music, sound effects, and/or the like.

The computing device may determine a classification for the firstportion. For example, the computing device may determine aclassification for a scene within the first portion based on thecontextual information. The computing device may determine one or moresuggestions for contextually relevant information for the first user.For example, the one or more suggestions for contextually relevantinformation may include one or more messages, one or more graphicalelements, such as symbols (e.g., emojis), graphics, text or videomessages, one or more pictures, and/or the like that are related to thecontent. For ease of explanation, the phrases “graphical symbol” and“graphical symbols” may be used herein. However, it is to be understoodthat any graphical symbol(s) described herein may comprise any graphicalelement, graphic, message, picture, etc., capable of conveyingcontextually relevant information. The computing device may determinethe one or more suggestions for contextually relevant information basedon the contextual information and/or the classification of the firstportion. The one or more suggestions may be based on the classificationfor the first portion. The classification may be based on an audiocomponent of the first portion, a visual component of the first portion,a textual component of the first portion and/or commentary dataassociated with the first portion satisfying (e.g., meeting orexceeding) a threshold.

The one or more suggestions may be determined by the computing devicebased on a user parameter associated with the first user, such as agraphical symbol usage history, a messaging history, an interactionhistory, a combination thereof, and/or the like. The parameter may also,or in the alternative, contain information about the user, such age,gender, or other descriptive attributes. The computing device may sendthe one or more suggestions for contextually relevant information to thefirst user device. The computing device may send the one or moresuggestions with the first portion of the content item. The first userdevice may receive the one or more suggestions and the first portion ofthe content item. The first user device may output at least one of theone or more suggestions and the first portion of the content item. Forexample, the first user device may output (e.g., display/show/present)the first portion of the content item and an overlay that includes atleast one of the one or more suggestions.

The first user may select at least one of the one or more suggestionsduring output of the first portion. For example, the at least onesuggestion may be a smiley face emoji, and the first user may select thesmiley face emoji during output of the first portion. The first userdevice may send an indication of the selection of the at least onesuggestion to the computing device. The first user device may send atimestamp associated with the first portion to the computing device. Thetimestamp may be indicative of a timecode associated with the selectionof the at least one suggestion. The computing device may receive theindication of the selection of the at least one suggestion and thetimestamp. The computing device may store the indication of theselection of the at least one suggestion and the timestamp as metadata.The metadata may include the indication of the selection of the at leastone suggestion (e.g., the smiley face emoji), the timestamp, anidentifier for the first user, an identifier for the first user device,and/or an identifier for the content item/first portion thereof. Thecomputing device may use the metadata when determining (e.g., selecting)one or more suggestions for contextually relevant information for thefirst user, another user, etc.

In some examples, the one or more suggestions may be provided at amobile device (e.g., a smartphone, tablet, etc.) in communication withthe first user device and/or associated with the first user. Thecomputing device may send the one or more suggestions for contextuallyrelevant information to the mobile device. The mobile device may output(e.g., display/show/present) the one or more suggestions with—orwithout—the first portion of the content item. The one or moresuggestions may be output with the first portion of the content item oran indication thereof (e.g., a frame, a title, etc.). For example, themobile device may output (e.g., display/show/present) the one or moresuggestions as an overlay along with the first portion of the contentitem. The first user may select at least one of the one or moresuggestions via the mobile device. The mobile device may send anindication of the selection of the at least one suggestion to thecomputing device. The mobile device may send a timestamp associated withthe first portion to the computing device. The timestamp may beindicative of a timecode associated with the selection of the at leastone suggestion via the mobile device. The computing device may receivethe indication of the selection of the at least one suggestion and thetimestamp from the mobile device. The computing device may store theindication of the selection of the at least one suggestion and thetimestamp as metadata. As discussed herein, the computing device may usethe metadata when determining (e.g., selecting) one or more suggestionsfor contextually relevant information for the first user, another user,etc.

Turning now to FIG. 1 , an example system 100 is shown. The system 100may comprise a content distribution network (CDN) 101 for contentdelivery, a first user device 104, a second user device 106, and a thirduser device 108. The CDN 101 may include a server 102. The server 102may be an edge server, a central office server, a headend, a nodeserver, a combination thereof, and/or the like. The CDN 101 may receivecontent (e.g., data, input programming, and/or the like) from multiplesources. The CDN 101 may combine the content from the various sourcesand may distribute the content to user devices (e.g., the first userdevice 104, the second user device 106, and/or the third user device 108via a network 110.

The CDN 101 may receive content from a variety of sources. The network110 may be configured to send content from a variety of sources using avariety of network paths, protocols, devices, and/or the like. Thenetwork 110 may be managed (e.g., deployed, serviced) by a contentprovider, a service provider, and/or the like. The network 110 may havea plurality of communication links connecting a plurality of devices.The network 110 may distribute signals from the CDN 101 to user devices,such as the first user device 104, the second user device 108, and/orthe third user device 108. The network 110 may be an optical fibernetwork, a coaxial cable network, a hybrid fiber-coaxial network, awireless network, a satellite system, a direct broadcast system, anEthernet network, a high-definition multimedia interface network, aUniversal Serial Bus (USB) network, or any combination thereof.

The first user device 104, the second user device 108, and/or the thirduser device 108 may be a set-top box, a digital streaming device, agaming device, a media storage device, a digital recording device, acomputing device, a mobile computing device (e.g., a laptop, asmartphone, a tablet, etc.), a combination thereof, and/or the like. Thefirst user device 104, the second user device 108, and/or the third userdevice 108 may implement one or more applications, such as contentviewers, social media applications, news applications, gamingapplications, content stores, electronic program guides, and/or thelike. The server 102 may provide services related to content and/orapplications. The server 102 may have an application store. Theapplication store may be configured to allow users to purchase,download, install, upgrade, and/or otherwise manage applications. Theserver 102 may be configured to allow users to download applications toa device, such as the first user device 104 and/or the second userdevice 108. The applications may enable a user of the first user device104, the second user device 108, and/or the third user device 108 tobrowse and select content items from a program guide. The server 102 mayrun one or more application services to send data, handle requests,and/or otherwise facilitate operation of applications for a user of thefirst user device 104, the second user device 108, and/or the third userdevice 108.

The server 102 may be in communication with one or more content sources103,105 of the CDN 101. The one or more content sources 103,105 may bemanaged by third party content providers, service providers, onlinecontent providers, over-the-top content providers, and/or the like.Content may be provided by the one or more content sources 103,105 via asubscription, by individual item purchase or rental, and/or the like.The one or more content sources 103,105 may be configured to sendcontent (e.g., video, audio, games, applications, data) to the firstuser device 104, the second user device 108, and/or the third userdevice 108 via the server 102 and the network 110.

Users of the first user device 104, the second user device 108, and/orthe third user device 108 may desire to provide feedback relating to acontent item they are presently viewing. As shown in FIG. 1 , the firstuser device 104 may request a news program and one or more suggestionsfor contextually relevant information may be provided at the first userdevice 104 along with the news program. As shown in FIG. 1 , the seconduser device 106 may request a movie (or any other type of content) andone or more suggestions for contextually relevant information may beprovided at the second user device 106 along with the movie. As shown inFIG. 1 , the third user device 108 may request a standup comedy specialand one or more suggestions for contextually relevant information may beprovided at the third user device 108 along with the standup comedyspecial. The one or more suggestions may be provided as an overlay106A,106B,108B including a message, a picture, an emoji, and/or thelike.

To provide contextually relevant information related to a content item,each of the first user device 104, the second user device 108, and/orthe third user device 108 may analyze a requested content item. As anexample, a first user of the first user device 104 may wish to view anews program. The first user device 104 may send a request for the newsprogram to the server 102 and/or the one or more content sources 103,105via the network 110. For purposes of explanation, the description hereinwill indicate the server 102 as the device that receives the request forthe news program. The server 102 may receive the request and analyze afirst portion of the news program. The first portion may be a fragment,a segment, a scene, etc. The server 102 may analyze the first portionusing a contextual module 202, as shown in FIG. 2 . The one or morecontent sources 103,105 may also include a contextual module 202 foranalyzing content items. The contextual module 202 may comprise an audioanalysis module 204, a visual analysis module 206, a captions analysismodule 208, and a commentary module 210.

The server 102 may analyze the first portion using the contextual module202 to determine contextual information associated with the firstportion. For example, the server 102 may use the audio analysis module204, the visual analysis module 206, the captions analysis module 208,and/or the commentary module 210 to determine an audio component of thefirst portion, a visual component of the first portion, a textualcomponent of the first portion, or commentary data associated with thefirst portion. An audio component of the first portion may include oneor more words spoken, music, sound effects, and/or the like. A visualcomponent of the first portion may include a person(s), a location, anobject(s), and/or the like. For example, the visual analysis module 206may perform facial recognition to determine an actor(s) depicted in aframe(s) of the first portion. A textual component of the first portionmay include one or more words spoken that are indicated by closedcaptioning or one or more words depicted in a frame(s) analyzed by thecaptions analysis module 208. Commentary data associated with the firstportion may comprise any data associated with commentary and/or opinionrelating to the content item or a portion thereof. The commentary datamay be stored at, or otherwise accessible by, the commentary module 210.The commentary data may comprise posts/messages associated with a socialmedia provider/platform, posts/messages associated with a contentprovider/platform, reviews (e.g., scores, articles, headlines, etc.),articles (or portions thereof), keywords, hashtags, a combinationthereof, and/or the like. The commentary module 210 may associate thecommentary data with the content item (or a portion thereof) based onthe data itself (e.g., words, images, etc.) and/or related metadata(e.g., identifiers, words, etc., associated with the contentitem/portion).

The contextual information may include one or more semantic concepts. Asdescribed herein, the audio analysis module 204, the visual analysismodule 206, the captions analysis module 208, and/or the commentarymodule 210 of the contextual module 202 may be used to determine anaudio component of the first portion, a visual component of the firstportion, a textual component of the first portion, or commentary dataassociated with the first portion. The contextual module 202 maydetermine one or more semantic concepts are associated with one or moreof the audio component of the first portion, the visual component of thefirst portion, or the textual component of the first portion. The one ormore semantic concepts may correspond to one or more image features(e.g., people, objects, landscape features, etc.) depicted in one ormore frames of the first portion. An image feature may be associatedwith multiple semantic concepts (e.g., at different levels ofgeneralization or categories, synonyms, etc.). For example, an imagefeature may be an identified celebrity actor, such as ArnoldSchwarzenegger, and the image feature may be associated with semanticconcepts including “governor,” “bodybuilder,” “action star,” etc. Theone or more semantic concepts may also include actions (e.g., walking,running, eating, etc.), emotions or moods (e.g., based on facialexpressions depicted in the frame(s)), a geographical location(s), orother topics related to the image feature.

The contextual module 202 may use a knowledge base (e.g., a knowledgegraph) and/or other databases of data that include a taxonomy ofconceptual terms arranged in a hierarchical graph of nodes to indicaterelationships between the one or more semantic concepts. The taxonomymay include objects (e.g., vehicles, animals, weapons, etc.), scenicfeatures (e.g., beach, sunset, etc.), actions (e.g., running, talking,kissing, etc.), emotions (e.g., anger, happiness, etc.), events (e.g.,fight, birthday, etc.), abstract concepts (e.g., revenge), a combinationthereof and/or the like. For example, the image feature may be acelebrity actor, such as Arnold Schwarzenegger, and the contextualmodule 202 may determine conceptual terms related to ArnoldSchwarzenegger, such as movies in which he stars. The contextualinformation may include the one or more semantic concepts and/or theconceptual terms.

As shown in FIG. 3A, the visual analysis module 206 may perform facialrecognition to determine actor depicted at specific timestamps of thefirst portion. As shown in FIG. 3B, the visual analysis module 206 mayperform a frame-by-frame analysis of the first portion to determine oneor more video events associated with each frame and/or timestamp of thefirst portion. For example, the frame-by-frame analysis of the firstportion may indicate people within the frame are drinking, kissing,eating, etc. The frame-by-frame analysis of the first portion mayindicate a car chase, an explosion, a baseball game, a basketball game,people fishing, etc. A textual component of the first portion analyzedby the captions analysis module 208 may include one or more words spokenthat are indicated by closed captioning or one or more words depicted ina frame(s) of the first portion. Commentary data associated with thefirst portion may comprise any data associated with commentary and/oropinion relating to the content item or a portion thereof.

The server 102 may determine a classification for the first portion(e.g., a scene). For example, the server 102 may determine aclassification for a scene within the first portion based on thecontextual information. The classification may include one or more of anemotion classification, a genre classification, a rating classification,a violence classification, or a chronological classification. Forexample, the first portion of the news program requested by the firstuser device 104 may be assigned an emotion classification of “none/notapplicable,” a genre classification of “news/documentary,” a ratingclassification of “general,” a violence classification of “none/notapplicable,” and/or a chronological classification of “first/opening,”etc.

The server 102 may determine the one or more suggestions forcontextually relevant information for the first user. For example, theone or more suggestions for contextually relevant information mayinclude one or more messages, one or more graphical elements, such assymbols (e.g., emojis), graphics, text or video messages, one or morepictures, and/or the like that are related to the content. For ease ofexplanation, the phrases “graphical symbol” and “graphical symbols” maybe used herein. However, it is to be understood that any graphicalsymbol(s) described herein may comprise any graphical element, graphic,message, picture, etc., capable of conveying contextually relevantinformation. The server 102 may determine the one or more suggestionsfor contextually relevant information based on the contextualinformation and/or the classification of the first portion. The one ormore suggestions may be based on the classification for the firstportion. The classification may be based on an audio component of thefirst portion, a visual component of the first portion, a textualcomponent of the first portion and/or commentary data associated withthe first portion satisfying (e.g., meeting or exceeding) a threshold.For example, the classification for the first portion—and the one ormore suggestions as a result—may be determined by the server 102 basedon one or more words spoken, music, sound effects, commentary data,and/or the like satisfying the threshold. As another example, theclassification for the first portion may be determined by the server 102based on a number of appearances and/or a duration that a specificperson(s) is present within the first portion, a specific location isdepicted within the first portion, a number of appearances and/or aduration that a specific object(s) is present within the first portion,and/or the like. As another example, the classification for the firstportion may be determined by the computing device based on one or morewords spoken satisfying the threshold. As a further example, theclassification of the first portion may be determined by the computingdevice based on commentary data (e.g., posts/messages, reviews,keywords, articles, hashtags, etc.) associated with the first portionsatisfying the threshold.

The one or more suggestions may be determined by the server 102 based ona user parameter associated with the first user. For example, the firstuser device 104 may send the user parameter to the server 102 as part ofthe request for the news program. The user parameter may be associatedwith a graphical symbol usage history, such as one or more emojis thatthe first user has previously used when providing feedback for contentitems. The user parameter may be associated with a messaging history,such as one or more messages the first user has previously sent whenproviding feedback for content items. The user parameter may beassociated with an interaction history, such as a history of previouslyused messages and/or graphical symbols that the first user haspreviously sent to a specific other user when providing feedback forcontent items.

The server 102 may send the one or more suggestions for contextuallyrelevant information to the first user device 104. The server 102 maysend the one or more suggestions with the first portion of the newsprogram. The first user device 104 may receive the one or moresuggestions and the first portion. The first user device 104 may outputat least one of the one or more suggestions as an overlay 104A with thefirst portion. For example, the first user device 104 may output (e.g.,display/show/present) the first portion and the overlay 104A may includeat least one of the one or more suggestions. The first user device 104may determine a static placement for the overlay and/or a dynamicplacement for the overlay. A static placement for the overlay 104A mayinclude the one or more suggestions displayed as graphical symbols at abottom of a screen of the first user device 104. A dynamic placement forthe overlay 104A may include the one or more suggestions displayed as afirst set of graphical symbols at the bottom of the screen of the firstuser device 104 during a first part of the first portion and a secondset of graphical symbols at the bottom of the screen of the first userdevice during a second part of the first portion.

The server 102 may determine a classification for multiple portions of acontent item. For example, the server 102 may perform a frame-by-frameand/or a scene-by-scene analysis of a content item (e.g., aframe-by-frame and/or a scene-by-scene classification). Based on theframe-by-frame and/or the scene-by-scene analysis, the server 102 mayprovide a plurality of sets of graphical symbols. Each set of theplurality of sets of the graphical symbols may be displayed while acorresponding scene and/or frame is displayed/output. For example, theserver 102 may determine a first classification for a first portion(e.g., a scene) and a second classification for a second portion (e.g.,another scene). The server 102 may determine a first set of one or moresuggestions for contextually relevant information based on the firstclassification for the first portion, and the server 102 may determine asecond set of one or more suggestions for contextually relevantinformation based on the second classification for the second portion.For example, a first portion of a content item requested via the seconduser device 106 may be an opening scene of an action movie where theantagonist of the movie is discussing a heist. The opening scene mayhave a greater amount of dialog as compared to other scenes of theaction movie. The server 102 may assign a first classification for thefirst portion. The first classification may include, for example, anemotion classification of “intriguing, etc.,” a genre classification of“action,” a rating classification of “general,” a violenceclassification of “none/not applicable,” and/or a chronologicalclassification of “first/opening,” etc.

The server 102 may determine one or more suggestions for contextuallyrelevant information for the user of the second user device 106 that arerelevant to the first portion (e.g., the opening scene). For example,the one or more suggestions for contextually relevant information forthe first portion may include one or more messages, one or moregraphical symbols (e.g., emojis), one or more pictures, and/or the likethat are related to the first portion of the content (e.g., suggestionsthat indicate a level of intrigue or interest). The server 102 maydetermine the one or more suggestions for contextually relevantinformation for the first portion based on the first classification. Thefirst classification for the first portion—and the one or moresuggestions for the first portion—may be based on an audio component ofthe first portion, a visual component of the first portion, a textualcomponent of the first portion and/or commentary data associated withthe first portion satisfying (e.g., meeting or exceeding) a threshold.The one or more suggestions for the first portion may be determined bythe server 102 based on a user parameter associated with the seconduser.

The server 102 may send the one or more suggestions for the firstportion to the second user device 106. The server 102 may send the oneor more suggestions for the first portion along with the first portionof the action movie. The second user device 106 may receive the one ormore suggestions for the first portion and the first portion of theaction movie. The second user device 106 may output (e.g.,display/show/present) the first portion of the action movie and anoverlay of at least one of the one or more suggestions for the firstportion.

The second user device 106 may request a second portion (e.g., anotherscene) of the content item (e.g., the action movie). The server 102 maydetermine a second set of one or more suggestions for contextuallyrelevant information based on a second classification for the secondportion. For example, as shown at the second user device 106 in FIG. 1 ,the second portion of the content item may be another scene of theaction movie where the antagonist of the movie causes an explosion tooccur. The other scene may have little or no dialog as compared to theopening scene of the action movie. The server 102 may assign the secondclassification for the second portion based on the explosion appearingin the second portion and/or based on a lesser amount of dialog ascompared to the first portion (e.g., the opening scene). The secondclassification may include, for example, an emotion classification of“exciting, tense, etc.” a genre classification of “action,” a ratingclassification of “teen, mature, etc.,” a violence classification of“medium, etc.,” and/or a chronological classification of “climax, apex,etc.”

The server 102 may determine one or more suggestions for contextuallyrelevant information for the user of the second user device 106 that arerelevant to the second portion. For example, the one or more suggestionsfor contextually relevant information for the second portion may includeone or more messages, one or more graphical symbols (e.g., emojis), oneor more pictures, and/or the like that are related to the first portionof the content (e.g., suggestions that indicate a high level ofexcitement, action, etc.). The server 102 may determine the one or moresuggestions for contextually relevant information for the second portionbased on the second classification. The second classification—and theone or more suggestions for the second portion as a result—may be basedon an audio component of the second portion (e.g., an explosion), avisual component of the second portion (e.g., an explosive cloud), atextual component of the second portion, and/or commentary dataassociated with the first portion satisfying (e.g., meeting orexceeding) a threshold. The one or more suggestions for the secondportion may be determined by the server 102 based on a user parameterassociated with the second user.

The server 102 may send the one or more suggestions for the secondportion to the second user device 106. The server 102 may send the oneor more suggestions for the second portion along with the second portionof the action movie. The second user device 106 may receive the one ormore suggestions for the second portion and the second portion of theaction movie. The second user device 106 may output (e.g.,display/show/present) the second portion of the action movie and theoverlay 106A. As shown in FIG. 1 , the overlay 106A may include at leastone of the one or more suggestions for the second portion (e.g.,contextually related to the scene of the action movie with theexplosion).

Turning now to FIG. 4A, an example user device 400 and an example mobiledevice 402 are shown. The user device 400 may be the first user device104, the second user device 108, and/or the third user device 108 shownin FIG. 1 . The user device 400 and the mobile device 402 may beconfigured to output content, to provide contextually relevantinformation associated with content items, and/or to allow a user tointeract with contextually relevant information. For example, a user ofthe user device 400 may request a standup comedy special. The userdevice 400 may send a request for the standup comedy special to a CDN,such as the CDN 101 of the system 100. The CDN may analyze a firstportion of the standup comedy special and determine (e.g., select) oneor more suggestions for contextually relevant information in a similarmanner as described herein within respect to the user device 104 and thenews program. The CDN may provide the one or more suggestions and thefirst portion to the user device 400. The user device 400 may output(e.g., display/show/present) the first portion and an overlay thatincludes at least one of the one or more suggestions. The user of theuser device 400 may select at least one 400A of the one or moresuggestions during output of the first portion. For example, the atleast one suggestion 400A may be a smiley face emoji, and the user mayselect the smiley face emoji during output of the first portion. Theuser device 400 may receive an indication of the selection of the atleast one suggestion 400A. The user device 400 may send the indicationof the selection of the at least one suggestion 400A and a timestampassociated with the selection to the CDN. For example, the user device400 may send the indication of the selection of the at least onesuggestion 400A and the timestamp associated with the selection to theserver 102 and/or the one or more content sources 103,105. For purposesof explanation, the description herein will indicate the server 102 asthe device that receives the indication of the selection of the at leastone suggestion 400A and the timestamp. The timestamp may be indicativeof a timecode associated with the selection of the at least onesuggestion 400A.

The user device 400 may be in communication with the mobile device 402via a network 404. The user device 400 and/or the mobile device 402 maybe in communication with other user devices and/or other mobile devicesvia the network 404. The network 404 may be an optical fiber network, acoaxial cable network, a hybrid fiber-coaxial network, a wirelessnetwork, a satellite system, a direct broadcast system, an Ethernetnetwork, a high-definition multimedia interface network, a UniversalSerial Bus (USB) network, or any combination thereof. Data may be sentby or to any of the devices shown in FIGS. 4A and 4B via the network406, including wireless paths (e.g., satellite paths, Wi-Fi paths,cellular paths, etc.) and terrestrial paths (e.g., wired paths, a directfeed source via a direct line, etc.).

In some examples, the one or more suggestions may be provided at themobile device 402, which may be a smartphone, a tablet, a laptop, etc.The server 102 may send the one or more suggestions for contextuallyrelevant information to the mobile device 402. The mobile device 402 mayoutput (e.g., display/show/present) the one or more suggestions with—orwithout—the first portion of the content item. For example, the mobiledevice 402 may output the one or more suggestions via a mobileapplication executing on the mobile device 402, such as a messaging app,a social media app, a media player app, a web browser, and/or the like.The one or more suggestions may be output with the first portion of thecontent item or an indication thereof (e.g., a frame, a title, etc.).For example, as shown in FIG. 4A, the mobile device 402 may output(e.g., display/show/present) the one or more suggestions as an overlay.The user may select the at least one suggestion 400A via the mobiledevice 402. The mobile device 402 may send an indication of theselection of the at least one suggestion 400A to the server 102. Themobile device 402 may send a timestamp associated with the first portionto the server 102. The timestamp may be indicative of a timecodeassociated with the selection of the at least one suggestion 400A viathe mobile device 402. The server 102 may receive the indication of theselection of the at least one suggestion 400A and the timestamp from themobile device 402. As discussed herein, the server 102 may use themetadata when determining (e.g., selecting) one or more suggestions forcontextually relevant information for the user, another user, etc.

The server 102 may store the indication of the selection of the at leastone suggestion 400A and the timestamp (e.g., received via the userdevice 400 and/or the mobile device 402) as metadata. The metadata mayinclude the indication of the selection of the at least one suggestion400A (e.g., the smiley face emoji), the timestamp, an identifier for theuser, an identifier for the user device 400, and/or an identifier forthe standup comedy special/first portion thereof. The server 102 may usethe metadata when determining (e.g., selecting) one or more suggestionsfor contextually relevant information for the user of the user device400A and/or for another user, etc.

As shown in FIG. 4B, a user device 401 and a mobile device 405 may be incommunication via the network 404. The user device 401 and the mobiledevice 405 may be in communication with the user device 400 and/or themobile device 402 via the network 404. The user device 401 and themobile device 405 may be configured to output content, to providecontextually relevant information associated with content items, and/orto allow a user to interact with contextually relevant information. Forexample, a user of the user device 401 a may request a content item. Theuser device 401 may send a request for the content item to the server102. The server 102 may determine that the content item requested by theuser of the user device 401 is the standup comedy special requested bythe user of the user device 400. The server 102 may determine that theuser of the user device 400 and the user of the user device 401 areassociated (e.g., based on corresponding user profiles). The server 102may determine (e.g., select) one or more suggestions 401B forcontextually relevant information for the user of the user device 401based on the metadata. The one or more suggestions 401B may include theat least one suggestion 400A (e.g., the smiley face emoji) previouslyselected by the user of the user device 400. The server 102 may send theone or more suggestions 401B to the user device 401.

The user device 401 may output the one or more suggestions 401B and thefirst portion of the standup comedy special. For example, the userdevice 401 may output (e.g., display/show/present) the first portion andan overlay that includes the one or more suggestions 401B. The userdevice 401 may provide a second overlay 401A. For example, the secondoverlay 401A may include the at least one suggestion 400A (e.g., thesmiley face emoji) previously selected by the user of the user device400. The user device 401 may provide the second overlay 401A duringoutput of the first portion at a timestamp that corresponds to thetimestamp associated with the selection of the at least one suggestion400A by the user of the user device 400. The timestamp associated withthe selection of the at least one suggestion 400A may correspond to apunchline of a funny joke told during the first portion of the standupcomedy special. In this way, the user of the user device 400 may providecontextually relevant information to the user of the user device 401.

In some examples, the server 102 may send the one or more suggestions401B to the mobile device 405. The mobile device 405 may be asmartphone, a tablet, a laptop, etc. The mobile device 405 may output(e.g., display/show/present) the one or more suggestions 401B with—orwithout—the first portion of the content item. For example, the mobiledevice 405 may output the one or more suggestions 401B via a mobileapplication executing on the mobile device 405, such as a messaging app,a social media app, a media player app, a web browser, and/or the like.As shown in FIG. 4B, the one or more suggestions 401B may be output withthe first portion of the content item or an indication thereof (e.g., aframe, a title, etc.). For example, as shown in FIG. 4B, the mobiledevice 402 may output (e.g., display/show/present) the one or moresuggestions 401B within the second overlay 401A. The second overlay 401Aoutput via the mobile device 405 may include the at least one suggestion400A (e.g., the smiley face emoji) previously selected by the user ofthe user device 400/the mobile device 402. The mobile device 405 mayprovide the second overlay 401A during output of the first portion at atimestamp that corresponds to the timestamp associated with theselection of the at least one suggestion 400A by the user of the userdevice 400/the mobile device 402. As shown in FIG. 4B, the secondoverlay 401A may be output with a frame (or other indication of) thefirst portion of the content item corresponding to the timestampassociated with the selection of the at least one suggestion 400A by theuser of the user device 400/the mobile device 402. In this way, the userof the user device 400 may provide contextually relevant information tothe user of the user device 401 via the mobile device 405. The user ofthe mobile device 405 may select at least one suggestion of the one ormore suggestions 401B via the mobile device 405. The mobile device 405may send an indication of the selection to the server 102. The mobiledevice 405 may send a timestamp associated with the first portion to theserver 102. The timestamp may be indicative of a timecode associatedwith the selection of the at least one suggestion of the one or moresuggestions 401B. The server 102 may receive the indication of theselection and the timestamp from the mobile device 405. The server 102may store the indication of the selection and the timestamp receivedfrom the mobile device 405 as metadata, which may be used whendetermining (e.g., selecting) one or more suggestions for contextuallyrelevant information.

Turning now to FIG. 5A, a user device 500 is shown. As described herein,the server 102 may analyze a portion of a content item using thecontextual module 202 and determine contextual information associatedwith the portion. For example, the server 102 may use the audio analysismodule 204, the visual analysis module 206, the captions analysis module208, and/or the commentary module 210 to determine an audio component ofthe first portion, a visual component of the first portion, a textualcomponent of the first portion, or commentary data associated with thefirst portion. For example, the user device 500 may send a request for acontent item, and the visual analysis module 206 may analyze one or moreimage features (e.g., people, objects, landscape features, etc.)depicted in one or more frames of the first portion. As shown in FIG.5A, a frame of the first portion of the content item may depict a personeating a lot of junk food. The visual analysis module 206 may analyzeone or more image features in the frames of the first portion todetermine one or more visual components, such as food, a person, etc.The commentary module 210 may analyze commentary data (or relatedmetadata) associated with the first portion. For example, the commentarymodule 210 may analyze posts/messages, reviews, articles, keywords,hashtags, etc., associated with the first portion. The contextual module202 may determine a semantic concept(s) associated with the one or morevisual components and/or the commentary data. For example, the semanticconcept(s) may correspond to the one or more image features, the one ormore visual components depicted in the frame, and/or the commentary dataassociated with the first portion shown in FIG. 5A. The semanticconcept(s) may be “eating,” “junk food,” etc.

The contextual module 202 may use the knowledge base described herein todetermine conceptual terms related to the semantic concept(s). Theserver 102 may determine one or more suggestions 502 for contextuallyrelevant information for a user of the user device 500. For example, theone or more suggestions 502 for contextually relevant information mayinclude one or more graphical elements, such as symbols (e.g., emojis),graphics, text or video messages, etc., that are related to the contentitem. For ease of explanation, the phrases “graphical symbol” and“graphical symbols” may be used herein. However, it is to be understoodthat any graphical symbol(s) described herein may comprise any graphicalelement, graphic, message, picture, etc., capable of conveyingcontextually relevant information. The server 102 may determine the oneor more suggestions 502 for contextually relevant information based onthe contextual information and/or the semantic concept(s). For example,the semantic concept(s) may be “eating,” “junk food,” etc., based on theframe of the first portion showing the person eating a lot of junk food.The one or more suggestions 502 may include a plurality of emojis thatare related to the semantic concept(s). For example, a suggestion 502Amay be an emoji of a pig (e.g., based on the semantic concept(s) “junkfood”). A suggestion 502B may be an emoji of a hamburger (e.g., based onthe semantic concept(s) “eating” and/or a visual component of the firstportion). A suggestion 502C may be an emoji indicating laughter/hilarity(e.g., based on the semantic concept(s) “junk food” and/or a visualcomponent of the first portion). The user device 500 may output the oneor more suggestions 502 and the first portion of the content item. Forexample, the user device 500 may output (e.g., display/show/present) thefirst portion and an overlay that includes the one or more suggestions502.

Turning now to FIG. 5B, a user device 501 is shown. As described herein,the server 102 may analyze a portion of a content item using thecontextual module 202 and determine contextual information associatedwith the portion. For example, the user device 501 may send a requestfor a content item, and the visual analysis module 206 may analyze oneor more image features depicted in one or more frames of the firstportion. As shown in FIG. 5B, a frame of the first portion of thecontent item may depict a man and a woman kissing. The visual analysismodule 206 may analyze one or more image features in the frames of thefirst portion to determine one or more visual components, such as a man,a woman, lips, etc. The contextual module 202 may determine a semanticconcept(s) associated with the one or more visual components. Forexample, the semantic concept(s) may correspond to the one or more imagefeatures, the one or more visual components depicted in the frame of thefirst portion shown in FIG. 5B, and/or the commentary data associatedwith the first portion. The semantic concept(s) may be “kissing,”“love,” etc. The contextual module 202 may use the knowledge basedescribed herein to determine conceptual terms related to the semanticconcept(s). The server 102 may determine one or more suggestions 503 forcontextually relevant information for a user of the user device 501. Forexample, the one or more suggestions 503 for contextually relevantinformation may include one or more graphical elements, such as symbols(e.g., emojis), graphics, text or video messages, etc., that are relatedto the content item. For ease of explanation, the phrases “graphicalsymbol” and “graphical symbols” may be used herein. However, it is to beunderstood that any graphical symbol(s) described herein may compriseany graphical element, graphic, message, picture, etc., capable ofconveying contextually relevant information. The server 102 maydetermine the one or more suggestions 502 for contextually relevantinformation based on the contextual information and/or the semanticconcept(s). For example, the semantic concept(s) may be “kissing,”“love,” etc., based on the frame of the first portion showing the manand the woman kissing. The one or more suggestions 503 may include aplurality of emojis that are related to the semantic concept(s). Forexample, a suggestion 503A may be an emoji of indicating love/romance(e.g., based on the semantic concept(s) “kissing” and/or “love”). Asuggestion 503B may be an emoji of kissing face (e.g., based on thesemantic concept(s) “kissing” and/or “love”).

As described herein, the one or more suggestions may be determined bythe server 102 based on a user parameter associated with the first user.For example, the user device 501 may send the user parameter to theserver 102 as part of the request for the content item. The userparameter may be associated with a graphical symbol usage history, suchas one or more emojis that a user of the user device 501 has previouslyused when providing feedback for content items. The server 102 maydetermine that the content item shown in FIG. 5B includes the semanticconcept(s) “kissing” and/or “love.” The server 102 may determine basedon the user parameter that the user of the user device 501 haspreviously selected an emoji indicating disgust/vomiting when viewingother content items having the same semantic concept(s). As anotherexample, the user parameter may indicate demographic information relatedto the user, such as age, gender, location, etc. The user parameter forthe user of the user device 501 may indicate the user is a 10 year oldmale. The server 102 may use the knowledge base described herein todetermine that 10 year old males typically select the emoji indicatingdisgust/vomiting when viewing other content items having the samesemantic concept(s). A suggestion 503C may be based on the userparameter. For example, the suggestion 503C may be the emoji indicatingdisgust/vomiting (e.g., based on prior use of that emoji when viewingrelated content items and/or based on the demographic information). Theuser device 501 may output the one or more suggestions 503 and the firstportion of the content item. For example, the user device 501 may output(e.g., display/show/present) the first portion and an overlay thatincludes the one or more suggestions 503.

As discussed herein, the present methods and systems may becomputer-implemented. FIG. 6 shows a block diagram depicting anenvironment 600 comprising non-limiting examples of a computing device601 and a server 602 connected through a network 604, such as thenetwork 110. The computing device 601 and/or the server 602 may be anyone of the devices shown in FIGS. 1 and 4 . In an aspect, some or allsteps of any described method herein may be performed on a computingdevice as described herein. The computing device 601 can comprise one ormultiple computers configured to store content data, such as contextualinformation, classifications, and the like. The server 602 can compriseone or multiple computers configured to store content data 624. Multipleservers 602 can communicate with the computing device 601 via thethrough the network 604.

The computing device 601 and the server 602 may each be a digitalcomputer that, in terms of hardware architecture, generally includes aprocessor 608, memory system 610, input/output (I/O) interfaces 612, andnetwork interfaces 614. These components (508, 610, 612, and 614) arecommunicatively coupled via a local interface 616. The local interface616 can be, for example, but not limited to, one or more buses or otherwired or wireless connections, as is known in the art. The localinterface 616 can have additional elements, which are omitted forsimplicity, such as controllers, buffers (caches), drivers, repeaters,and receivers, to enable communications. Further, the local interfacemay include address, control, and/or data connections to enableappropriate communications among the aforementioned components.

The processor 608 can be a hardware device for executing software,particularly that stored in memory system 610. The processor 608 can beany custom made or commercially available processor, a centralprocessing unit (CPU), an auxiliary processor among several processorsassociated with the computing device 601 and the server 602, asemiconductor-based microprocessor (in the form of a microchip or chipset), or generally any device for executing software instructions. Whenthe computing device 601 and/or the server 602 is in operation, theprocessor 608 can be configured to execute software stored within thememory system 610, to communicate data to and from the memory system610, and to generally control operations of the computing device 601 andthe server 602 pursuant to the software.

The I/O interfaces 612 can be used to receive user input from, and/orfor providing system output to, one or more devices or components. Userinput can be provided via, for example, a keyboard and/or a mouse.System output can be provided via a display device and a printer (notshown). I/O interfaces 612 can include, for example, a serial port, aparallel port, a Small Computer System Interface (SCSI), an infrared(IR) interface, a radio frequency (RF) interface, and/or a universalserial bus (USB) interface.

The network interface 614 can be used to transmit and receive from thecomputing device 601 and/or the server 602 on the network 604. Thenetwork interface 614 may include, for example, a 10BaseT EthernetAdaptor, a 100BaseT Ethernet Adaptor, a LAN PHY Ethernet Adaptor, aToken Ring Adaptor, a wireless network adapter (e.g., WiFi, cellular,satellite), or any other suitable network interface device. The networkinterface 614 may include address, control, and/or data connections toenable appropriate communications on the network 604.

The memory system 610 can include any one or combination of volatilememory elements (e.g., random access memory (RAM, such as DRAM, SRAM,SDRAM, etc.)) and nonvolatile memory elements (e.g., ROM, hard drive,tape, CDROM, DVDROM, etc.). Moreover, the memory system 610 mayincorporate electronic, magnetic, optical, and/or other types of storagemedia. Note that the memory system 610 can have a distributedarchitecture, where various components are situated remote from oneanother, but can be accessed by the processor 608.

The software in memory system 610 may include one or more softwareprograms, each of which comprises an ordered listing of executableinstructions for implementing logical functions. In the example of FIG.6 , the software in the memory system 610 of the computing device 601can comprise a suitable operating system (O/S) 618. In the example ofFIG. 6 , the software in the memory system 610 of the server 602 cancomprise, the content data 624, and a suitable operating system (O/S)618. The operating system 618 essentially controls the execution ofother computer programs and provides scheduling, input-output control,file and data management, memory management, and communication controland related services.

For purposes of illustration, application programs and other executableprogram components such as the operating system 618 are illustratedherein as discrete blocks, although it is recognized that such programsand components can reside at various times in different storagecomponents of the computing device 601 and/or the server 602. Animplementation of the contextual module 202 can be stored on ortransmitted across some form of computer readable media. Any of thedisclosed methods can be performed by computer readable instructionsembodied on computer readable media. Computer readable media can be anyavailable media that can be accessed by a computer. By way of exampleand not meant to be limiting, computer readable media can comprise“computer storage media” and “communications media.” “Computer storagemedia” can comprise volatile and non-volatile, removable andnon-removable media implemented in any methods or technology for storageof information such as computer readable instructions, data structures,program modules, or other data. Exemplary computer storage media cancomprise RAM, ROM, EEPROM, flash memory or other memory technology,CD-ROM, digital versatile disks (DVD) or other optical storage, magneticcassettes, magnetic tape, magnetic disk storage or other magneticstorage devices, or any other medium which can be used to store thedesired information and which can be accessed by a computer.

Turning now to FIG. 7 , a flowchart of an example method 700 forproviding contextually relevant information is shown. The method 700 maybe performed in whole or in part by a single computing device, aplurality of computing devices, and the like. For example, the firstuser device 104, the second user device 106, the third user device 108,the server 102, and/or the one or more content sources 103,105 of thesystem 100 may be configured to perform the method 700. As anotherexample, the contextual module 202, the user device 400, the user device401, the user device 500, and/or the user device 501 may be configuredto perform the method 700. As a further example, the computing device601 or the server 602 of the system 600 may be configured to perform themethod 700.

A first computing device, such as the server 102, may receive a requestfor a content item. The request for the content item may be receivedfrom a user via a second computing device, such as a user device, amobile device, etc. The first computing device may analyze at least oneportion of the content item based on the request. For example, the firstcomputing device may perform a portion-by-portion analysis of thecontent item, such as a frame-by-frame and/or a scene-by-scene analysis.Based on the portion-by-portion analysis, the first computing device mayprovide a plurality of sets of graphical symbols. Each set of theplurality of sets of the graphical symbols may be displayed while acorresponding portion of the content item is displayed/output.

At step 702, the first computing device may determine contextualinformation associated with a portion of the content item. Thecontextual information may be based on the portion of the content item.For example, the first computing device may use one or more contextualmodules, such as the contextual module 202, to determine the contextualinformation based on at least one of: an audio component of the portionof the content item, a visual component of the portion of the contentitem, a textual component of the portion of the content item, orcommentary data associated with the portion of the content item. Anaudio component of the portion of the content item may include one ormore words spoken, music, sound effects, and/or the like. A visualcomponent of the portion of the content item may include a person(s), alocation, an object(s), and/or the like. For example, the visualanalysis module may perform facial recognition to determine an actor(s)depicted in a frame(s) of the portion of the content item. A textualcomponent of the portion of the content item may include one or morewords spoken that are indicated by closed captioning or one or morewords depicted in a frame(s) of the portion of the content item.Commentary data associated with the portion may comprise any dataassociated with commentary and/or opinion relating to the content itemor a portion thereof. The commentary data may comprise posts/messagesassociated with a social media provider/platform, posts/messagesassociated with a content provider/platform, reviews (e.g., scores,articles, headlines, etc.), articles (or portions thereof), keywords,hashtags, a combination thereof, and/or the like.

The first computing device may determine a classification for theportion (e.g., a scene). For example, the first computing device maydetermine a classification for a scene within the portion based on thecontextual information. The classification may include one or more of anemotion classification, a genre classification, a rating classification,a violence classification, or a chronological classification. Forexample, the content item may be a romantic comedy, and a scene withinthe portion may be assigned an emotion classification of “funny,” agenre classification of “romantic comedy,” a rating classification of“mature,” a violence classification of “none/not applicable,” and/or achronological classification of “first/opening,” etc.

At step 704, the first computing device may determine a plurality ofgraphical symbols. For ease of explanation, the phrases “graphicalsymbol” and “graphical symbols” may be used herein. However, it is to beunderstood that any graphical symbol(s) described herein may compriseany graphical element, graphic, message, picture, etc., capable ofconveying contextually relevant information. For example, the firstcomputing device may determine one or more suggestions for contextuallyrelevant information for the first user. The one or more suggestions mayinclude the plurality of graphical symbols. The first computing devicemay determine the plurality of graphical symbols based on the contextualinformation and/or the classification of the portion of the contentitem. The plurality of graphical symbols may be based on theclassification for the portion. The classification may be based on anaudio component of the portion, a visual component of the portion, atextual component of the portion, and/or commentary data associated withthe portion of the content item satisfying (e.g., meeting or exceeding)a threshold. For example, the classification—and the plurality ofgraphical symbols as a result—may be determined by the first computingdevice based on one or more words spoken, music, sound effects, and/orthe like satisfying the threshold. As another example, theclassification may be determined by the first computing device based ona number of appearances and/or a duration that a specific person(s) ispresent within the portion, a specific location is depicted within theportion, a number of appearances and/or a duration that a specificobject(s) is present within the portion, and/or the like. As anotherexample, the classification may be determined by the first computingdevice based on one or more words spoken satisfying the threshold. As afurther example, the classification of the first portion may bedetermined by the computing device based on commentary data (e.g.,posts/messages, reviews, keywords, articles, hashtags, etc.) associatedwith the first portion satisfying the threshold.

The plurality of graphical symbols may be determined by the firstcomputing device based on a user parameter associated with the firstuser. For example, the second computing device may send the userparameter to the first computing device as part of the request for thecontent item. The user parameter may be associated with a graphicalsymbol usage history, such as one or more emojis that the user haspreviously used when providing feedback for content items. The userparameter may be associated with a messaging history, such as one ormore messages the user has previously sent when providing feedback forcontent items. The user parameter may be associated with an interactionhistory, such as a history of previously used messages and/or graphicalsymbols that the user has previously sent to a specific other user whenproviding feedback for content items.

At step 706, the first computing device may cause output of at least aportion of the plurality of graphical symbols with the portion of thecontent item. For example, the first computing device may send theplurality of graphical symbols to the second computing device. The firstcomputing device may send the plurality of graphical symbols with theportion of the content item. The second computing device may receive theplurality of graphical symbols and the portion of the content item. Thesecond computing device may output at least one of the plurality ofgraphical symbols with—or without—the portion of the content item. Forexample, the second computing device may output (e.g.,display/show/present) the portion of the content item and an overlaythat includes at least one of the plurality of graphical symbols. Thesecond computing device may determine a static placement for the overlayand/or a dynamic placement for the overlay. As described herein, thefirst computing device may perform a portion-by-portion analysis of thecontent item, such as a frame-by-frame and/or a scene-by-scene analysis.Based on the portion-by-portion analysis, the first computing device mayprovide the static placement and/or the dynamic placement for theoverlay comprising the at least one graphical symbol while acorresponding portion of the content item is displayed/output. Forexample, the static placement for the overlay may include the at leastone graphical symbol displayed at a bottom of a screen of the secondcomputing device. As another example, the dynamic placement for theoverlay may include the at least one graphical symbol displayed as afirst set of the plurality of graphical symbols at the bottom of thescreen of the second computing device during a first part of the portionand a second set of the plurality of graphical symbols at the bottom ofthe screen of the second computing device during a second part of theportion.

The user may select at least one of the plurality of graphical symbolsduring output of the portion. For example, the at least one graphicalsymbol may be a smiley face emoji, and the user may select the smileyface emoji during output of the portion. The second computing device mayreceive an indication of the selection of the at least one graphicalsymbol. The second computing device may send the indication of theselection of the at least one graphical symbol to the first computingdevice. The second computing device may send a timestamp associated withthe portion to the first computing device. The timestamp may beindicative of a timecode associated with the selection of the at leastone graphical symbol. The first computing device may receive theindication of the selection of the at least one graphical symbol and thetimestamp.

The first computing device may store the indication of the selection ofthe at least one graphical symbol and the timestamp as metadata. Themetadata may include the indication of the selection (e.g., the smileyface emoji), the timestamp, an identifier for the user, an identifierfor the second computing device, and/or an identifier for the contentitem/portion thereof. The first computing device may use the metadatawhen determining (e.g., selecting) one or more suggestions forcontextually relevant information (e.g., a plurality of graphicalsymbols) for the user of the second computing device, another user, etc.For example, a user of a third computing device may request a contentitem. The user of the third computing device may send a request for thecontent item to the first computing device. The first computing devicemay determine that the content item requested by the user of the thirdcomputing device is the same content item requested by the user (e.g.,select) of the second computing device. The first computing device maydetermine that the user of the second computing device and the user ofthe third computing device are associated (e.g., based on correspondinguser profiles).

Based on the content item requested by the user of the third computingdevice being the same content item requested by the user of the secondcomputing device and/or based on the association between the users, thefirst computing device may determine (e.g., select) one or moresuggestions for contextually relevant information (e.g., a plurality ofgraphical symbols) for the user of the third computing device based onthe metadata. The one or more suggestions for contextually relevantinformation for the user of the third computing device may be a secondplurality of graphical symbols. The second plurality of graphicalsymbols may include the at least one graphical symbol (e.g., the smileyface emoji) previously selected by the user of the second computingdevice. The first computing device may send the second plurality ofgraphical symbols to the third user device.

Turning now to FIG. 8 , a flowchart of an example method 800 forproviding contextually relevant information is shown. The method 800 maybe performed in whole or in part by a single computing device, aplurality of computing devices, and the like. For example, the firstuser device 104, the second user device 106, the third user device 108,the server 102, and/or the one or more content sources 103,105 of thesystem 100 may be configured to perform the method 800. As anotherexample, the contextual module 202, the user device 400, the user device401, the user device 500, and/or the user device 501 may be configuredto perform the method 800. As a further example, the computing device601 or the server 602 of the system 600 may be configured to perform themethod 800.

At step 802, a first computing device, such as a user device, a mobiledevice, etc., may send at least one user parameter to a second computingdevice, such as the server 102. The first computing device may send theat least one user parameter as part of a request for a content item. Thesecond computing device may receive the request for the content item andthe at least one user parameter. The request for the content item may bereceived from a user via a first computing device, such as a userdevice, a mobile device, etc.

The second computing device may analyze a portion of the content itembased on the request. The second computing device may determinecontextual information associated with the portion of the content item.The contextual information may be based on the portion of the contentitem. For example, the second computing device may use one or morecontextual modules, such as the contextual module 202, to determine thecontextual information based on at least one of: an audio component ofthe portion of the content item, a visual component of the portion ofthe content item, a textual component of the portion of the contentitem, or commentary data associated with the portion of the contentitem. An audio component of the portion of the content item may includeone or more words spoken, music, sound effects, and/or the like. Avisual component of the portion of the content item may include aperson(s), a location, an object(s), and/or the like. For example, thevisual analysis module may perform facial recognition to determine anactor(s) depicted in a frame(s) of the portion of the content item. Atextual component of the portion of the content item may include one ormore words spoken that are indicated by closed captioning or one or morewords depicted in a frame(s) of the portion of the content item.Commentary data associated with the portion may comprise any dataassociated with commentary and/or opinion relating to the content itemor a portion thereof. The commentary data may comprise posts/messagesassociated with a social media provider/platform, posts/messagesassociated with a content provider/platform, reviews (e.g., scores,articles, headlines, etc.), articles (or portions thereof), keywords,hashtags, a combination thereof, and/or the like.

The second computing device may determine a classification for theportion (e.g., a scene). For example, the second computing device maydetermine a classification for a scene within the portion based on thecontextual information. The classification may include one or more of anemotion classification, a genre classification, a rating classification,a violence classification, or a chronological classification. Forexample, the content item may be a romantic comedy, and a scene withinthe portion may be assigned an emotion classification of “funny,” agenre classification of “romantic comedy,” a rating classification of“mature,” a violence classification of “none/not applicable,” and/or achronological classification of “first/opening,” etc.

The second computing device may determine a plurality of graphicalsymbols. For ease of explanation, the phrases “graphical symbol” and“graphical symbols” may be used herein. However, it is to be understoodthat any graphical symbol(s) described herein may comprise any graphicalelement, graphic, message, picture, etc., capable of conveyingcontextually relevant information. For example, the second computingdevice may determine one or more suggestions for contextually relevantinformation for the first user. The one or more suggestions may includethe plurality of graphical symbols (e.g., emojis). The second computingdevice may determine the plurality of graphical symbols based on thecontextual information and/or the classification of the portion of thecontent item. The plurality of graphical symbols may be based on theclassification for the portion. The classification may be based on anaudio component of the portion, a visual component of the portion, atextual component of the portion, and/or commentary data associated withthe portion of the content item satisfying (e.g., meeting or exceeding)a threshold. For example, the classification—and the plurality ofgraphical symbols as a result—may be determined by the second computingdevice based on one or more words spoken, music, sound effects, and/orthe like satisfying the threshold. As another example, theclassification may be determined by the second computing device based ona number of appearances and/or a duration that a specific person(s) ispresent within the portion, a specific location is depicted within theportion, a number of appearances and/or a duration that a specificobject(s) is present within the portion, and/or the like. As anotherexample, the classification may be determined by the second computingdevice based on one or more words spoken satisfying the threshold. As afurther example, the classification of the first portion may bedetermined by the computing device based on commentary data (e.g.,posts/messages, reviews, keywords, articles, hashtags, etc.) associatedwith the first portion satisfying the threshold.

The plurality of graphical symbols may be determined by the secondcomputing device based on the at least one user parameter associatedwith the user of the first computing device. The at least one userparameter may be associated with a graphical symbol usage history, suchas one or more emojis that the user has previously used when providingfeedback for content items. The at least one user parameter may beassociated with a messaging history, such as one or more messages theuser has previously sent when providing feedback for content items. Theat least one user parameter may be associated with an interactionhistory, such as a history of previously used messages and/or graphicalsymbols that the user has previously sent to a specific other user whenproviding feedback for content items.

At step 804, the first computing device may receive the portion of thecontent item. For example, the second computing device may send theportion of the content item to the first computing device. At step 806,the first computing device may receive the plurality of graphicalsymbols. For example, the second computing device may send the pluralityof graphical symbols to the first computing device. The second computingdevice may send the plurality of graphical symbols with the portion ofthe content item. The first computing device may receive the pluralityof graphical symbols and the portion of the content item. At step 808,the first computing device may output at least one of the plurality ofgraphical symbols and the portion of the content item. For example, thefirst computing device may output (e.g., display/show/present) theportion of the content item and an overlay that includes at least one ofthe plurality of graphical symbols. The first computing device maydetermine a static placement for the overlay and/or a dynamic placementfor the overlay. As described herein, the first computing device mayperform a portion-by-portion analysis of the content item, such as aframe-by-frame and/or a scene-by-scene analysis. Based on theportion-by-portion analysis, the first computing device may provide thestatic placement and/or the dynamic placement for the overlay comprisingthe at least one graphical symbol while a corresponding portion of thecontent item is displayed/output. For example, the static placement forthe overlay may include the at least one graphical symbol displayed at abottom of a screen of the first computing device. As another example,the dynamic placement for the overlay may include the at least onegraphical symbol displayed as a first set of the plurality of graphicalsymbols at the bottom of the screen of the first computing device duringa first part of the portion and a second set of the plurality ofgraphical symbols at the bottom of the screen of the first computingdevice during a second part of the portion.

The user may select at least one of the plurality of graphical symbolsduring output of the portion. For example, the at least one graphicalsymbol may be a smiley face emoji, and the user may select the smileyface emoji during output of the portion. The first computing device mayreceive an indication of the selection of the at least one graphicalsymbol. The first computing device may send the indication of theselection of the at least one graphical symbol to the second computingdevice. The first computing device may send a timestamp associated withthe portion to the second computing device. The timestamp may beindicative of a timecode associated with the selection of the at leastone graphical symbol. The second computing device may receive theindication of the selection of the at least one graphical symbol and thetimestamp.

The second computing device may store the indication of the selection ofthe at least one graphical symbol and the timestamp as metadata. Themetadata may include the indication of the selection (e.g., the smileyface emoji), the timestamp, an identifier for the user, an identifierfor the first computing device, and/or an identifier for the contentitem/portion thereof. The second computing device may use the metadatawhen determining (e.g., selecting) one or more suggestions forcontextually relevant information (e.g., a plurality of graphicalsymbols) for the user of the first computing device, another user, etc.For example, a user of a third computing device (e.g., a user device, amobile device, etc.) may request a content item. The user of the thirdcomputing device may send a request for the content item to the secondcomputing device. The second computing device may determine that thecontent item requested by the user of the third computing device is thesame content item requested by the user (e.g., select) of the firstcomputing device. The second computing device may determine that theuser of the first computing device and the user of the third computingdevice are associated (e.g., based on corresponding user profiles).

Based on the content item requested by the user of the third computingdevice being the same content item requested by the user of the firstcomputing device and/or based on the association between the users, thesecond computing device may determine (e.g., select) one or moresuggestions for contextually relevant information (e.g., a plurality ofgraphical symbols) for the user of the third computing device based onthe metadata. The one or more suggestions for contextually relevantinformation for the user of the third computing device may be a secondplurality of graphical symbols. The second plurality of graphicalsymbols may include the at least one graphical symbol (e.g., the smileyface emoji) previously selected by the user of the first computingdevice. The second computing device may send the second plurality ofgraphical symbols to the third user device.

Turning now to FIG. 9 , a flowchart of an example method 900 forproviding contextually relevant information is shown. The method 900 maybe performed in whole or in part by a single computing device, aplurality of computing devices, and the like. For example, the firstuser device 104, the second user device 106, the third user device 108,the server 102, and/or the one or more content sources 103,105 of thesystem 100 may be configured to perform the method 900. As anotherexample, the contextual module 202, the user device 400, the user device401, the user device 500, and/or the user device 501 may be configuredto perform the method 900. As a further example, the computing device601 or the server 602 of the system 600 may be configured to perform themethod 900.

A first computing device, such as the server 102, may receive a firstrequest for a content item. The first request for the content item maybe received from a user via a second computing device, such as a userdevice, a mobile device, etc. The first computing device may analyze aportion of the content item based on the request. At step 902, the firstcomputing device may determine contextual information associated withthe portion of the content item. The contextual information may be basedon the portion of the content item. For example, the first computingdevice may use one or more contextual modules, such as the contextualmodule 202, to determine the contextual information based on at leastone of: an audio component of the portion of the content item, a visualcomponent of the portion of the content item, a textual component of theportion of the content item, or commentary data associated with theportion of the content item. An audio component of the portion of thecontent item may include one or more words spoken, music, sound effects,and/or the like. A visual component of the portion of the content itemmay include a person(s), a location, an object(s), and/or the like. Forexample, the visual analysis module may perform facial recognition todetermine an actor(s) depicted in a frame(s) of the portion of thecontent item. A textual component of the portion of the content item mayinclude one or more words spoken that are indicated by closed captioningor one or more words depicted in a frame(s) of the portion of thecontent item. Commentary data associated with the portion may compriseany data associated with commentary and/or opinion relating to thecontent item or a portion thereof. The commentary data may compriseposts/messages associated with a social media provider/platform,posts/messages associated with a content provider/platform, reviews(e.g., scores, articles, headlines, etc.), articles (or portionsthereof), keywords, hashtags, a combination thereof, and/or the like.

The first computing device may determine a classification for theportion (e.g., a scene). For example, the first computing device maydetermine a classification for a scene within the portion based on thecontextual information. The classification may include one or more of anemotion classification, a genre classification, a rating classification,a violence classification, or a chronological classification. Forexample, the content item may be a romantic comedy, and a scene withinthe portion may be assigned an emotion classification of “funny,” agenre classification of “romantic comedy,” a rating classification of“mature,” a violence classification of “none/not applicable,” and/or achronological classification of “first/opening,” etc.

For ease of explanation, the phrases “graphical symbol” and “graphicalsymbols” may be used herein. However, it is to be understood that anygraphical symbol(s) described herein may comprise any graphical element,graphic, message, picture, etc., capable of conveying contextuallyrelevant information. The first computing device may determine aplurality of graphical symbols. For example, the first computing devicemay determine one or more suggestions for contextually relevantinformation for the first user. The one or more suggestions may includethe plurality of graphical symbols (e.g., emojis). The first computingdevice may determine the plurality of graphical symbols based on thecontextual information and/or the classification of the portion of thecontent item. The plurality of graphical symbols may be based on theclassification for the portion. The classification may be based on anaudio component of the portion, a visual component of the portion, atextual component of the portion, and/or commentary data associated withthe portion of the content item satisfying (e.g., meeting or exceeding)a threshold. For example, the classification—and the plurality ofgraphical symbols as a result—may be determined by the first computingdevice based on one or more words spoken, music, sound effects, and/orthe like satisfying the threshold. As another example, theclassification may be determined by the first computing device based ona number of appearances and/or a duration that a specific person(s) ispresent within the portion, a specific location is depicted within theportion, a number of appearances and/or a duration that a specificobject(s) is present within the portion, and/or the like. As anotherexample, the classification may be determined by the first computingdevice based on one or more words spoken satisfying the threshold. As afurther example, the classification of the first portion may bedetermined by the computing device based on commentary data (e.g.,posts/messages, reviews, keywords, articles, hashtags, etc.) associatedwith the first portion satisfying the threshold.

The plurality of graphical symbols may be determined by the firstcomputing device based on a user parameter associated with the firstuser. For example, the second computing device may send the userparameter to the first computing device as part of the first request forthe content item. The user parameter may be associated with a graphicalsymbol usage history, such as one or more emojis that the user haspreviously used when providing feedback for content items. The userparameter may be associated with a messaging history, such as one ormore messages the user has previously sent when providing feedback forcontent items. The user parameter may be associated with an interactionhistory, such as a history of previously used messages and/or graphicalsymbols that the user has previously sent to a specific other user whenproviding feedback for content items.

The first computing device may cause output of at least a portion of theplurality of graphical symbols with the portion of the content item. Forexample, at step 904, the first computing device may send the pluralityof graphical symbols to the second computing device. The first computingdevice may send the plurality of graphical symbols with the portion ofthe content item. The second computing device may receive the pluralityof graphical symbols and the portion of the content item. The secondcomputing device may output at least one of the plurality of graphicalsymbols and the portion of the content item. For example, the secondcomputing device may output (e.g., display/show/present) the portion ofthe content item and an overlay that includes at least one of theplurality of graphical symbols. The second computing device maydetermine a static placement for the overlay and/or a dynamic placementfor the overlay. As described herein, the first computing device mayperform a portion-by-portion analysis of the content item, such as aframe-by-frame and/or a scene-by-scene analysis. Based on theportion-by-portion analysis, the first computing device may provide thestatic placement and/or the dynamic placement for the overlay comprisingthe at least one graphical symbol while a corresponding portion of thecontent item is displayed/output. For example, the static placement forthe overlay may include the first graphical symbol displayed at a bottomof a screen of the second computing device. As another example, thedynamic placement for the overlay may include the first graphical symboldisplayed as a first set of the plurality of graphical symbols at thebottom of the screen of the second computing device during a first partof the portion and a second set of the plurality of graphical symbols atthe bottom of the screen of the second computing device during a secondpart of the portion.

The user may select a first graphical symbol of the plurality ofgraphical symbols during output of the portion. For example, the firstgraphical symbol may be a smiley face emoji, and the user may select thesmiley face emoji during output of the portion. The second computingdevice may receive an indication of the selection of the first graphicalsymbol. The second computing device may send the indication of theselection of the first graphical symbol to the first computing device.At step 906, the first computing device may receive the indication ofthe selection of the first graphical symbol of the plurality ofgraphical symbols. The second computing device may send a timestampassociated with the portion to the first computing device. The timestampmay be indicative of a timecode associated with the selection of thefirst graphical symbol. The first computing device may receive thetimestamp with the indication of the selection of the first graphicalsymbol.

The first computing device may store the indication of the selection ofthe first graphical symbol and the timestamp as metadata. The metadatamay include the indication of the selection (e.g., the smiley faceemoji), the timestamp, an identifier for the user, an identifier for thesecond computing device, and/or an identifier for the contentitem/portion thereof. The first computing device may use the metadatawhen determining (e.g., selecting) one or more suggestions forcontextually relevant information (e.g., a plurality of graphicalsymbols) for the user of the second computing device, another user, etc.For example, a user of a third computing device (e.g., a user device, amobile device, etc.) may request a content item. The user device of thethird computing device may send a second request for the content item tothe first computing device. At step 908, the first computing device mayreceive the second request for the content item from the third computingdevice. The first computing device may determine that the content itemrequested by the user of the third computing device is the same contentitem requested by the user (e.g., select) of the second computingdevice. The first computing device may determine that the user of thesecond computing device and the user of the third computing device areassociated (e.g., based on corresponding user profiles).

Based on the content item requested by the user of the third computingdevice being the same content item requested by the user of the secondcomputing device and/or based on the association between the users, thefirst computing device may determine (e.g., select) one or moresuggestions for contextually relevant information (e.g., a plurality ofgraphical symbols) for the user of the third computing device based onthe metadata. The one or more suggestions for contextually relevantinformation for the user of the third computing device may be a secondplurality of graphical symbols. The second plurality of graphicalsymbols may include the first graphical symbol (e.g., the smiley faceemoji) previously selected by the user of the second computing device.At step 910, the first computing device may send the portion of thecontent item and at least the first graphical symbol to the thirdcomputing device. For example, the first computing device may send thefirst graphical symbol as part of the second plurality of graphicalsymbols.

Turning now to FIG. 10 , a flowchart of an example method 1000 forproviding contextually relevant information is shown. The method 1000may be performed in whole or in part by a single computing device, aplurality of computing devices, and the like. For example, the firstuser device 104, the second user device 106, the third user device 108,the server 102, and/or the one or more content sources 103,105 of thesystem 100 may be configured to perform the method 1000. As anotherexample, the contextual module 202, the user device 400, the user device401, the user device 500, and/or the user device 501 may be configuredto perform the method 1000. As a further example, the computing device601 or the server 602 of the system 600 may be configured to perform themethod 1000.

A computing device, such as the server 102, may receive a request for acontent item. The request for the content item may be received from auser via a second computing device, such as a user device, a mobiledevice, etc. At step 1002, the computing device may determine firstcontextual information associated with a first portion of the contentitem. For example, the first computing device may perform aportion-by-portion analysis of the content item, such as aframe-by-frame and/or a scene-by-scene analysis. Based on theframe-by-frame and/or the scene-by-scene analysis, at step 1004, thecomputing device may determine a first portion of a plurality ofgraphical symbols associated with the first portion. The first portionof the plurality of graphical symbols may be displayed/output/presentedwhile a corresponding scene and/or frame is displayed/output (e.g., at auser device). The computing device may determine a first classificationfor the first portion (e.g., a scene) and a second classification for asecond portion (e.g., another scene). The computing device may determinethe first classification based on the first contextual information. Atstep 1006, the computing device may determine second contextualinformation. For example, the computing device may determine the secondcontextual information based on the second classification for the secondportion (e.g., another scene).

At step 1008, the computing device may determine a second portion of theplurality of graphical symbols. For example, the computing device maydetermine the second portion of the plurality of graphical symbols basedon the second classification for the second portion. The first portionof the content item may be an opening scene of an action movie where theantagonist of the movie is discussing a heist. The opening scene mayhave a greater amount of dialog as compared to other scenes of theaction movie. The computing device may assign the first classificationfor the first portion. The first classification may include, forexample, an emotion classification of “intriguing, etc.,” a genreclassification of “action,” a rating classification of “general,” aviolence classification of “none/not applicable,” and/or a chronologicalclassification of “first/opening,” etc. The first classification for thefirst portion may be based on an audio component of the first portion, avisual component of the first portion, a textual component of the firstportion and/or commentary data associated with the first portionsatisfying (e.g., meeting or exceeding) a threshold.

The computing device may determine the second classification for thesecond portion. The second portion of the content item may be anotherscene of the action movie where the antagonist of the movie causes anexplosion to occur. The other scene may have little or no dialog ascompared to the opening scene of the action movie. The computing devicemay assign the second classification for the second portion based on theexplosion appearing in the second portion and/or based on a lesseramount of dialog as compared to the first portion (e.g., the openingscene). The second classification may include, for example, an emotionclassification of “exciting, tense, etc.” a genre classification of“action,” a rating classification of “teen, mature, etc.,” a violenceclassification of “medium, etc.,” and/or a chronological classificationof “climax, apex, etc.” The computing device may determine the secondportion of the plurality of graphical symbols based on the secondclassification. The second classification may be based on an audiocomponent of the second portion (e.g., an explosion), a visual componentof the second portion (e.g., an explosive cloud), a textual component ofthe second portion, and/or commentary data associated with the firstportion satisfying (e.g., meeting or exceeding) a threshold.

At step 1010, the computing device may cause presentation/output of thesecond portion of the plurality of graphical symbols. For example, thecomputing device may cause presentation/output of the second portion ofthe plurality of graphical symbols with the second portion of thecontent item. The computing device may cause presentation/output of thesecond portion of the plurality of graphical symbols at a user device.The user device may receive the second portion of the plurality ofgraphical symbols and the second portion of the content item. The userdevice may output at least one of the plurality of graphical symbolswith—or without—the second portion of the content item. For example, theuser device may output (e.g., display/show/present) the second portionof the content item and an overlay that includes at least one of theplurality of graphical symbols. The user device may determine a staticplacement for the overlay and/or a dynamic placement for the overlay. Asdescribed herein, the computing device may perform a portion-by-portionanalysis of the content item, such as a frame-by-frame and/or ascene-by-scene analysis. Based on the portion-by-portion analysis, thecomputing device may provide the static placement and/or the dynamicplacement for the overlay comprising the at least one graphical symbolwhile a corresponding portion of the content item is displayed/output.For example, the static placement for the overlay may include the atleast one graphical symbol displayed at a bottom of a screen of the userdevice. As another example, the dynamic placement for the overlay mayinclude the at least one graphical symbol displayed at the bottom of thescreen of the user device during the second portion of the content item.

While specific configurations have been described, it is not intendedthat the scope be limited to the particular configurations set forth, asthe configurations herein are intended in all respects to be possibleconfigurations rather than restrictive. Unless otherwise expresslystated, it is in no way intended that any method set forth herein beconstrued as requiring that its steps be performed in a specific order.Accordingly, where a method claim does not actually recite an order tobe followed by its steps or it is not otherwise specifically stated inthe claims or descriptions that the steps are to be limited to aspecific order, it is in no way intended that an order be inferred, inany respect. This holds for any possible non-express basis forinterpretation, including: matters of logic with respect to arrangementof steps or operational flow; plain meaning derived from grammaticalorganization or punctuation; the number or type of configurationsdescribed in the specification.

It will be apparent to those skilled in the art that variousmodifications and variations may be made without departing from thescope or spirit. Other configurations will be apparent to those skilledin the art from consideration of the specification and practicedescribed herein. It is intended that the specification and describedconfigurations be considered as exemplary only, with a true scope andspirit being indicated by the following claims.

1. A method comprising: sending, by a computing device, a userparameter; receiving, based on the user parameter, an element; andcausing presentation of the element.